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Motivation as a determinant in problem-based learning 

Abstract 

Problem-based learning is an approach to professional education that stresses the use of 
real life problems in education. Several attempts have been made to understand the intricacies 
involved in the process of problem-based learning. This paper adds to this work. It focuses the 
attention to the commitment of students to actually engage in the problem-based learning 
process. 

Subjects were approximately 1300 undergraduates. The data were analyzed using a 
structural equations modeling approach. The following variables, besides commitment, were 
investigated: Group functioning, tutor functioning, prior knowledge, the quality of the 
instructional materials, time spent on study, achievement and increased interest. 

As hypothesized, commitment turns out a potent determinant of achievement. In fact, it 
represents one of the strongest relationships in the model. 



Introduction 

Problem-based learning is an approach to professional education that stresses the use 
of real life problems in education. Students work in small tutorial groups and, while 
discussing these problems, formulate goals for self-directed learning. The learning resulting 
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from these activities is considered constructive and contextually meaningful (Barrows, 1983; 
Norman & Schmidt, 1992). 

This educational approach has methodological consequences for the organization of 
the student's learning process. Each year of study is sub-divided into periods of six weeks. 
These periods of six weeks are called “blocks”. For the period of a block, the students are 
concerned with one theme. For this purpose a group of staff members compiles what is called 
a “block book”. This book presents the problems concerning the theme of the block. There is 
also a bibliography of the literature that may be studied and the names and telephone numbers 
of staff members who can be consulted, and furthermore indications as to the possible appro- 
aches that students could adopt with the problems. 

The students then start working on the basis of the problems in the block book. They 
meet together for a few hours twice a week in an educational group made up of about eight to 
ten fellow students of the same class. The educational group is arbitrarily assembled and its 
composition changes after each block. Each group has a member of staff as a “tutor”. The 
tutor has the task of keeping an eye on the quality of the work within the group and promoting 
it, without concerning himself with the content of the discussions. He or she is therefore not 
the person who provides answers if the students are unable to do so. The tutor guides the 
educational group in purposeful cooperation in the set problems. 

The tutorial method has some advantages (Pallie & Carr, 1987; Barrows et al., 1 986). 

In the first place, the students are, in following this method, continuously being trained in 
dealing with and “learning to solve problems”. When they have left university, they will forget 
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many of the facts that they learned there, and, furthermore, a great deal of what they learned 
will soon be out of date. Skill in solving problems efficiently and in gaining access to new, 
relevant information is therefore of considerable importance. 

Problem-based education appears to have a strong motivating effect too. The students feel that 
they are involved with reality and not that they just have to learn “dry” theory. The students 
learn social and communicative skills in verbal interaction with others. Furthermore regular 
group meetings provide better opportunities for making social contacts than is the case with a 
great mass of students attending lectures (Van Berkel et al., 1995). 



The process in the educational group session 

The process in the educational group can be divided into seven phases. 

The students first ensure that all of the concepts that are used in this formulation are clear to 
everybody (clarifying) . When there is sufficient clarity among them as to the nature of the 
particular problem, they make an inventory of the knowledge that they already have within the 
group on this subject (activation of prior knowledge) . Various members of the group also put 
forward hypotheses: “couldn't it be that ....”. In this brainstorming phase, a number of aspects 
of the problem emerge, and a number of questions arise that cannot immediately be answered. 
The group then together determines which aspects they want to investigate more deeply 
(clustering) and they make the arrangements among themselves: “Let us study these and those 
points for the following session”. Less important matters or things that merely have to be 
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looked up are then divided among the group members (formulating the learning goals ). A 
group meeting lasting one or two hours ends with this type of agreement. 

In the coming days, the members of the group occupy themselves individually with 
their share of the task, with the help of handbooks, literature, etc. They can also go to the 
“open plan study area” for this purpose: This is a complex of study rooms where there is 
sufficient literature and other material to enable them to pursue their study fruitfully 
(individual study) . At the following group meeting an inventory is made of the results of this 
study (reporting the learning goals) . This will often lead to a group discussion. Two members 
of the group may, for example, arrive at different answers to the same question, or the material 
that has been studied may not provide sufficient clarity. Being critically active with the 
information the student has found, increases the depth of insight into the problem, it helps to 
develop deep processing and it is, in academic training, a productive means of promoting 
quality. Sometimes a staff member is asked to come and help the group with a problem with 
which they are unable to progress any further. If the group feels they have made sufficient pro- 
gress with a problem, although it has not been solved, it can be “shelved” and they can 
approach a new problem. 



Modeling the process of problem based learning 

Each new education process should prove itself on its value and merit. Several 
evaluation approaches have been developed to determine the effects of different educational 
programs. The classical approaches are concerned with questions such as “How much did 
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students learn?” and “Did students learn more compared to the traditional program?” These 
questions focus on the outcome of the educational process; they ignore the influences of the 
instructional and learning processes on the final student-learning results. As Lawton (1980) 
stated, evaluation must be concerned with the total context of an educational situation, with 
the causes and its result (Gijselaers & Schmidt, 1989). 

The theories tested have been formulated within the models-of-school-leaming 
tradition represented by authors like Carroll (1963), Bloom (1976) and Cooley and Leinhardt 
(1990). These investigators maintain that learning within the context of a school can be 
described by three categories of variables. The first category is that of the input variables, 
notably student characteristics, teacher behavior and the adequacy of the learning materials. 
The second category comprises the intervening process variables: Learning activities carried 
out by students, time spent on study and features of the instructional process. The third 
category finally consists of cognitive output variables, e.g. achievement, and affective 
outcomes like interest in subject-matter studied (Schmidt & Gijselaers, 1990). 

Several attempts have been made to understand the intricacies involved in the process 
of problem-based learning. In particular much research has focused on the question of what 
influences what in problem-based learning, for example, on the role of the tutor in problem- 
based learning. In particular, the issue whether her or his subject-matter expertise would be a 
help or a hindrance in the students’ learning process has been the subject of much debate (e.g. 
Silver & Wilkerson, 1991). In addition, review studies have been carried out trying to assess 
the influence of problem-based instruction versus conventional, lecture-based instruction on 
student achievement and other variables of interest (e.g. Vernon & Blake, 1993). 
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A different approach have been taken by Gijselaers and Schmidt (1989) and Schmidt 
and Gijselaers (1990). These authors have tried to model problem-based learning, applying 
recently developed structural equations modeling techniques (Bentler, 1989). In their theory, 
they distinguish between what they call input variables to the learning process, such as the 
student’s prior knowledge, the quality of the instructional materials, and the effectiveness of 
the tutor; learning process variables, such as the extent to which the tutorial group functions 
effectively, and the amount of time students spend on self-directed learning activities; and 
output variables such as the resulting achievement and increased interest in the topic studied. 
According to Gijselaers and Schmidt, prior knowledge, instructional materials and tutor all 
influence tutorial group functioning. Tutorial group functioning, in turn, influences time spent 
and interest, whereas time spent on learning influences the level of achievement. In a series of 
studies, they found empirical support for this model. 

In summary, figure 1 outlines a causal representation of the learning going on within a 
problem-based curriculum. 

According to the model, an increase in the magnitude of one of the variables 
characteristically causes an increase of the magnitudes of other variables. The arrows indicate 
the direction of the causal influences. 

This paper extends this work. It directs attention to a variable that seems to have been 
overlooked by most authors, namely the willingness, or commitment, of students to actually 
engage in the problem-based learning process. In fact, it seems to be implicitly assumed that 
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problem-based learning is so motivating in itself that students almost inevitably do what is 
required in these curricula (Boud & Feletti, 1997). Reality is different. Everyone involved in 
those programs knows about students who do not seem to be touched by the approach and 
continue their studies contributing little to the group’s work. In addition, dropout is not 
uncommon in these programs (although less common than in conventional programs). In the 
present study, the amount of commitment demonstrated by students was considered a 
determinant of their success in such programs. We operationalized commitment in terms of 
attendance of tutorial group meetings and hypothesized that the more the students attended, 
the better their achievement would be, simply because attendance could be read as a measure 
of personal engagement that students feel towards their study. It is important to note that 
students who do not attend the tutorial group, still study the learning goals. They ask their 
fellow students which learning goals they had formulated so they can prepare themselves for 
the next group meeting. 



Method 



Subjects 

Subjects were approximately 1300 undergraduates, enrolled in a problem-based, four- 
year, health sciences curriculum. The students worked in small tutorial groups of 
approximately ten students, guided by a tutor. The curriculum consisted of a series of 
consecutive courses or “units”, each lasting six weeks. At the end of an unit, students fill in an 
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evaluation form. The response rate on this form is 80%. For each of the academic years data 
from an average 30 of these units were included in the analysis. 

Instruments 

At the end of each unit, students were required to fill in a rating scale, consisting of 42 
Likert-type items, covering various dimensions of problem-based learning as outlined in 
figure 1 . Each item could be answered on a 5-point scale. In previous factor-analytic, 
reliability and validity studies, these dimensions have been shown to have good average 
intraclass-coefficients (as a measure of interrater agreement) and construct validity (Schmidt, 
Dolmans, Gijselaers, & Des Marchais, 1995). 

Amount of prior knowledge (“PRIORKN”) was estimated by asking students to what 
degree the unit’s subject-matter linked with their prior knowledge. Quality of the problems in 
the block book (“BLOCKB”) was measured by four items exploring to what extent the 
problems were clearly stated, stimulated group discussion, encouraged self-directed learning 
activities, etc. Tutor performance (“TUTORFUN”) was measured by a fourteen-item rating 
scale: How well was the tutor informed about the unit’s objectives? To what extent did he or 
she facilitate learning through stimulating questions? How much interest did he or she show in 
the students’ learning? Small-group functioning (“GROUPFUN”) was measured by two items 
inquiring about working-procedures, cooperation among group members, task-orientation of 
the group, etc. Time spent on learning (“TIME”) was measured by asking students to give an 
estimate of the number of hours per week spent on self-directed learning activities. An 
achievement (“ACHIEV”) test was administered to each student at the end of each unit. These 
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so called block tests consist of 200 items. Attendance (“ATTEND”) was measured by the 
tutor. Finally, intrinsic interest in subject-matter (“INTEREST”) was measured by two items. 

Procedure 

Four year groups comprising 1300 students were studied. Data however were aggregated 
at the tutorial group level because achievement and other related variables related to members 
of the same tutorial group cannot be considered independent scores (Marsh, 1980). Doing so, 
aggregated data would obscure the individual variation. 

Statistical analysis 

A path analysis was carried out to assess the adequacy of the model (Bollen, 1989). In 
this study a recursive path analytic model was used because only one-way causal influences 
are represented in the model. This procedure yields several statistics that allow the 
investigator to assess the extent to which the empirical data fit the theoretical model proposed. 
There is no single best statistic which gives insight into the fit of the model. The best a 
researcher can do, is to calculate several statistics. The coherence of these measures indicates 
the model-fit. 

Based on the simple chi-square and the degrees of freedom, a level of significance (P) is 
computed. In order not to reject the model, the value of P must be higher than .05. 

Arbuckle (1997) argues that the Cmin/df is the best index of fit. This index is computed by 
dividing the minimum discrepancy (C) by its degrees of freedom. The ratio should be close to 
1 for correct models. 
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Fo is an index which estimates the population discrepancy of the model. In general, the 
more the number of parameters in the model, the better the fit. Steiger (1980) suggested 
compensating for the effect of model complexity by the number of degrees of freedom. 

Taking the square root of the resulting ratio gives the population “root mean square error of 
approximation”, called RMSEA (Browne and Cudeck, 1993). Its value must be less than 
0 . 1000 . 

The goodness of fit index (GFI) is another index which indicates the fit of the model. 
Its value is between 0 and 1, where 1 indicates a perfect fit. 



Results 



Table 1 shows the intercorrelations, means and standard deviations of the variables. 
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There were 131 tutorial groups involved in this study. During one block period there are 
twelve meetings scheduled. In general, the meetings are well attended by the students (mean 
attendance percentage is 10.66/12 * 100 = 89%). The block test (“achievement”) was scored 
on a 4-point scale (1 = insufficient, 4 = excellent). 

There are four hypothesized dependent variables in this study: Attendance, small 
group functioning, time spent on study and achievement. It appears that there are relatively 
high significant correlations between attendance and small group functioning, amount of prior 
knowledge (negatively), time spent on study (negatively) and achievement. Small group 
functioning is highly correlated with intrinsic interest in subject matter, amount of prior 
knowledge, quality of the problems in the block book and achievement. Time spent on study 
shows high correlation with the quality of the problems in the block book. Achievement is 
highly correlated with attendance, small group functioning, and the quality of the problems in 
the block book. 

However, not all the significant intercorrelations were part of the path analysis. 

Based on the significance of the beta weights, computed by a multiple regression analysis 
(stepwise), only those paths were selected which contributed significantly. 

Figure 2 displays the major findings. Arrows indicate the hypothesized causal 
influences. Double arrows indicate correlations. The number beside the rectangles indicate the 
explained variance (R 2 ) which is of course dependent on influences of the independent 
variables. The following abbreviations were used in figure 2: 
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Achiev: 


Achievement 


Attend: 


Attendance 


Blockb: 


Quality of the problems in the block book 


Groupfun: 


Small group functioning 


Interest: 


Intrinsic interest in subject matter 


Priorkn: 


Amount of prior knowledge 


Time: 


Time spent on study 


Tutorfun: 


Tutor performance 
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Chi-square for this model is 13.944 (df= 1 1; p = .236). The level of significance 
indicates that the hypothesized model is not rejected. Other measures of model fit are also 
satisfactory. The minimum discrepancy, C, divided by the degrees of freedom (Cmin/df) = 

1 .268; square root of the population discrepancy (Fo) corrected by the complexity of the 
model (degrees of freedom) (Fo/df) RMSEA = .045; Goodness of fit index (GFI) = .974. 

To investigate the robustness of the model, the data set was divided into several 
subgroups. Firstly, the group (N = 131) was randomly split into two, group A and group B. 
Secondly, model fit was also computed in four groups. Each group represents the year of study 
(Table 2). 

It appeared that the model was not rejected for four of the six groups. 

Another analysis focuses on how the independent variables account for the 
variances of the dependent variables (see table 3). The percentages represents the amount of 
explained variance by the dependent variables. 

The variance of the variable attendance (R 2 = .26) is explained by amount of prior 
knowledge (29%), quality of the problems in the block book (27%) and small group 
functioning (44%). 



Conclusion and discussion 
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This research extends the work ofGijselaers and Schmidt (1990). They had 
formulated a theoretical model of problem-based learning, following the ideas of Carroll 
(1963). The model has three categories of variables: Input, intervening and cognitive output 
variables. This research adds a new variables to the model: The commitment of students to 
actually engage in the problem-based learning process. 

In general, the findings suggest that the hypothesized model fits the data adequately. 
There are some very interesting results. For example the negative causal relationship between 
the quality of the problems in the block book and attendance. It seems that poor quality of the 
problems leads to more attendance in the tutorial group. Apparently, students compensate a 
poor block book by attending the group. In contrast, a good block book leads to better 
functioning of the tutorial group, a high test result, and also to more time spent on study. 
Research into what makes a problem a good problem should be an area of primary interest to 
investigators in this field, as Schmidt and Gijselaers (1990) have already stated. Tutor 
performance has a relatively high influence ( .39) on the functioning of the group. The 
absence of a causal influence of time spent on study on achievement seems remarkable. It 
appears that the amount of study time does not significantly lead to higher test scores. 

The present findings concur reasonably well with the original Schmidt and Gijselaers 
findings. In fact, the addition of the new variable in the problem-based model (“attendance”) 
increases the fit substantial. There is, however, one remarkable difference from Schmidt and 
Gijselaers. In the research we conducted, the variable intrinsic interest in subject matter turns 
out to be a independent variable, instead of an dependent variable. It is not clear how this 
result can be explained. In the model tested in our research, the influence of this variable on 
time spent on study is substantially. Omitting this variable results in a less good fit (although 
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the model is still not rejected). Adjusting the model by introducing this variable as an 
independent variable leads to the model being rejected. So at least, our data suggest that the 
Schmidt-Gijselaers model is inappropriate in this respect. 

Compared to the theoretical model of problem-based learning (figure 1) there are other 
inconsistencies. There is no path between prior knowledge and group functioning. This 
suggests that, contrary to theoretical expectations, more prior knowledge among group 
members does not generally improve the functioning of the tutorial group. Second, the model 
fits well without a causal relationship between group functioning and time spent on study. The 
theoretical model assumes a positive (and thus strong) relationship. These inconsistencies 
were also found by Schmidt and Gijselaers (1990), so it seems that in this respect the original 
theoretical model need to be adjusted. 

As hypothesized, attendance of the tutorial group meetings turns out a potent 
determinant of achievement. Thus, the more students show their commitment to the work of 
their group by attending its meetings, the higher their achievement at the end of a unit. The 
(negative) relationship between attendance and study time is another indicator of this causal 
chain: Commitment to the group’s work translates itself into less intensive self-study and 
higher achievement. In a way, the groups are functioning as study groups so that students do a 
great deal of their learning in the group and can therefore do without additional self-study at 
the cost to their achievement levels. 

In conclusion: A real commitment to problem-based learning as expressed in the 
willingness to attend sessions seems to be a major factor in explaining achievement in this 
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kind of curriculum. Further research in this area might prove fruitful and is warranted. The 
results reported here suggest that there is a need to rebuild the theoretical problem-based 
learning model. 
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Figure 1 : Causal Model Of Problem-Based Learning 
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Table 1 : Intercorrelations. Means And Standard Deviations Of The Aggregated Scores On The 



Variables (N = 131) 

(Intercorrelations in italics were used as input in the path analysis) 





1 


2 


3 


4 


5 


6 


7 


8 


1 . Attendance 


- 
















2. Tutor 
performance 


.07 


- 














3. Small group 
functioning 


.21* 


.60* 


- 












4. Intrinsic 
interest in subject 
matter 


- .01 


.30* 


.51* 


“ 










5. Amount of 
prior knowledge 


-.36* 


.14 


.27* 


.38* 


- 








6. Time spent on 
study 


-.29 


.03 


.08 


.09 


.09 


- 






7. Quality of the 
problems in the 
block book 


-.17 


.45* 


,64* 


.73* 


.54* 


.30* 


“ 




8. Achievement 


.26* 


.12 


.18* 


.21 


.15 


.08 


.30* 


- 


Means 


1 .66 


55.37 


7.63 


7.83 


3.45 


17.20 


14.76 


3.04 


Standard dev. 


0.49 


5.19 


0.79 


0.82 


0.46 


3.74 


1.41 


0.23 


N of items 


1 


14 


2 


2 


1 


1 


4 


200 
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Table 2: Model Fit Of Several Subgroups 



Group 


N 


Chi-square 


Df 


P 


Cmin/df 


RMSEA 


GFI 


Total 


131 


13.944 


11 


.236 


1.268 


.045 


.974 


Group A 


60 


22.462 


11 


.021 


2.042 


.143 


.895 


Group B 


61 


8.859 


11 


.635 


0.805 


.000 


.997 


Year 1 


29 


10.644 


11 


.474 


0.968 


.000 


.926 


Year 2 


38 


7.582 


11 


.750 


0.689 


.000 


.961 


Year 3 


38 


26.625 


11 


.005 


2.420 


.196 


.884 


Year 4 


16 


7.452 


11 


.761 


0.677 


.000 


.896 



23 




24 



Motivation as a determinant in problem-based learning 



Table 3: Percentages Of Variance Explained (R 2 f 



Dependent variables 





Achievement 


Time spent 
on study 


Small 

group 

functioning 


Attendance 


Intrinsic interest in 
subject matter 




15% 






Amount of prior 
knowledge 




15% 




29% 


Quality of the problems 
in the block book 


53% 


35% 


54% 


27% 


Attendance 


47% 


35% 






Tutor performance 






46% 




Time spent on study 










Small group functioning 








44% 


Total R 2 


0.18 


0.20 


0.53 


0.26 
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Figure 2: Path model of problem-based learning 

26 



« 




TM031272 




U.S. Department of Education 

Office of Educational Research and Improvement 
(OERI) 

National Library of Education (NLE) 

Educational Resources Information Center (ERIC) 



Reproduction Release 

(Specific Document) 



I. DOCUMENT IDENTIFICATION: 



Motivation to commit oneself as a determinant of achievement in problem-based learning 



Title: 



Author(s): Henk J.M. van Berkel & Henk H. Schmidt 



Corporate Source: 



Publication Date: April 1999 



II. REPRODUCTION RELEASE: 

In order to disseminate as widely as possible timely and significant materials of interest to the educational 
community, documents announced in the monthly abstract journal of the ERIC system, Resources in Education 
(RIE), are usually made available to users in microfiche, reproduced paper copy, and electronic media, and sold 
through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if 
reproduction release is granted, one of the following notices is affixed to the document. 



ERjt 



If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following 
three options and sign in the indicated space following. 



The sample sticker shown below will be affixed to 
all Level 1 documents 


The sample sticker shown below will be affixed to all Level 2A 
documents 


The sample sticker sh 
all Level 2B documen 


PERMISSION TO REPRODUCE AND 
disseminate THIS MATERIAL HAS 
BEENORANffiDBY 


PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL IN 
MICROFICHE, AND IN ELECTRONIC MEDIA 
FOR ERIC COLLECTION SUBSCRIBERS ONLY, 
HAS BEEN ORAN^D BY 

/ 

TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


PERMISSION TO 
DISSEMINATE 1 
MICROFICHE ONLY F 







TO TOE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 


TO THE EDLldAT 
INFORMATION 


Level 1 


Level 2A 


Level 2B 


XX 


t 


t 






Check here for Level 1 release, permitting 
reproduction and dissemination in microfiche or 
other ERIC archival media (e.g. electronic) and 
paper copy. j 


Check here for Level 2A release, permitting reproduction and 
dissemination in microfiche and in electronic media for ERIC 
archival collection subscribers only j 


Check here for Level 
reproduction and diss 




Documents will be processed as indicated provided reproduction quality permits. 

If permission to reproduce is granted, but no box is checked, documents will be processed at Level 1. 



I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate 
indicated above. Reproduction from the ERIC microfiche, or electronic media by persons other than ERIC employees and its sy 
requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other service a 
information needs of educators in response to discrete inquiries. 


Signature: 7 — — — - — 


Printed Name/Position/Title; 






Henk J.M. van Berkel, PhD. 




Organization/ Address: 


Telephone: ..43 388 1121 


Fax: ... 43 388 4140 


Maastricht University 


E-mail Address: 


Date: November 9th 1999 


P.O. Box 616 
6200 MD Maastricht 
The Netherlands 


h.vanberkel@educ.unimaas.nl 





III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 



If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from 
another source, please provide the following information regarding the availability of the document. (ERIC will not 
announce a document unless it is publicly available, and a dependable source can be specified. Contributors should 
also be aware that ERIC selection criteria are significantly more stringent for documents that cannot be made 
available through EDRS.) 




IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the 
appropriate name and address; 



Name: 



Address: 



V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearinghouse: 


ERIC Clearinghouse on Assessment and Evaluation 
1129 Shriver Laboratory (Bldg 075) 

College Park, Maryland 20742 


Telephone: 301-405-7449 
Toll Free: 800-464-3742 
Fax: 301-405-8134 
ericae@ericae.net 
http://ericae.net 



EFF-088 (Rev. 9/97) 




